Using Optical Flow for Tracking

We present two observation models based on optical flow information to track objects using particle filter algorithms. Although, in principle, the optical flow information enables us to know the displacement of the objects present in a scene, it cannot be used directly to displace a model since flow estimation techniques lack the necessary precision. We will define instead two observation models for using into probabilistic tracking algorithms: the first uses an optical flow estimation computed previously, and the second is based directly on correlation techniques over two consecutive frames. 1 Probabilistic Tracking The probabilistic models applied to tracking [1,2,3] enable us to estimate the a posteriori probability distribution of the set of valid configurations for the object to be tracked, represented by a vector X, from the set of measurements Z taken from the images of the sequence, p(X|Z). The estimation in the previous instant is combined with a dynamical model giving rise to the a priori distribution in the current instant, p(X). The relation between these distributions is given by Bayes’ Theorem: p(X|Z) ∝ p(X) · p(Z|X) The distribution p(Z|X), known as the observation model, represents the probability of the measurements Z appearing in the images, assuming that a specific configuration of the model in the current instant is known. In this paper, two optical flow based observation models are defined. The first one uses as evidence an existing estimation of the optical flow of the sequence, and the second one is based on correlation techniques. A. Sanfeliu and J. Ruiz-Shulcloper (Eds.): CIARP 2003, LNCS 2905, pp. 87–94, 2003. c © Springer-Verlag Berlin Heidelberg 2003 88 M. Lucena, J.M. Fuertes, and N. Perez de la Blanca 2 Optical Flow Estimation The most well-known hypothesis for calculating the optical flow [4] assumes that the intensity structures found in the image, on a local level, remain approximately constant over time, at least during small intervals of time. There is no algorithm for estimating the optical flow field which is clearly superior to the others. Each may have small advantages over the others in particular situations, but in general it may be said that from a practical point of view all are equivalent [5,6]. In this paper, we have preferred to use the algorithm in [7], for the following reasons: – It does not impose restrictions on the sequence to be analyzed. – It provides a dense estimation of the optical flow. – It is designed to preserve discontinuities in the flow, which is necessary for the observation model proposed in this section to behave appropriately. 3 The Dynamical Model Other authors have successfully used characteristics such as the gradient [8] or intensity distributions [3] for tracking tasks. The dynamical model of the object will provide an a priori distribution on all the possible configurations in the instant tk, p(X(tk)), from the estimated distributions in the previous instants of time. In this paper, a second-order dynamical model has been used in which the two previous states of the object model are considered, and this is equivalent to taking a first-order dynamical model with a state vector for the instant tk of the form [8] Xtk = [Xtk−1 Xtk ] The integration of the a priori distribution p(X) with the set Z of the evidences present in each image, in order to obtain the a posteriori distribution p(X|Z), is obtained with Bayes’ Theorem. This fusion of information can be performed, if the distributions are Gaussian, using Kalman’s Filter [1]. However, in general, the distributions involved in the process are normally not Gaussian and multimodal [2]. Sampling methods for modeling this type of distribution [9] have shown themselves to be extremely useful, and particle filter algorithms [10, 11,3] based on sets of weighted random samples, enable their propagation to be performed effectively. 4 Observation Models If there is a good optical flow measurement and the object is perfectly localized, it is possible to slide the points of the model in accordance with the flow vectors, thereby obtaining a good estimate of its position for the following frame. Unfortunately, the small errors in the flow will mount up with each frame, so that the model gradually separates from the real object, until it loses it completely. Using Optical Flow for Tracking 89 Nevertheless, it may be supposed that the object to be tracked will move in an environment that has other displacements, and therefore it may be assumed that there will be discontinuities in the optical flow on its contour –or at least part of it. The observation model will be defined in such a way that it not only helps the flow inside the object to concur with the displacement implied by the value of X , but also so that discontinuities in the optical flow appear in the contour of the model. 4.1 Observation Model Based on Optical Flow Let us suppose that we have an estimation of the flow field v for the image I in the instant tk. The following error function may be defined, with S ⊆ I being an area inside the image: